GENERALIZATION OF THE SERIAL POSITION EFFECT ACROSS MODALITY AND STIMULUS TYPE
ORIANE DALEY, Y. ZHANG IE 532 Human Information Processing Spring 2012
ABSTRACT
The serial position effect (SPE) refers to the differences in recalled items as being a function of position in a list, with items positioned in the middle of a list recalled less compared to those positioned at the beginning and end of the list. This study generalizes the serial position effect across stimulus modality (visual vs. auditory) and stimulus type (verbal vs. spatial). In the first two blocks, participants were presented verbal stimuli either in auditory mode (Av), visual mode (Vv), or auditory plus visual stimuli (A/Vv) to compare the stimulus modality effect. In the third block, verbal stimuli (A/V v) and spatial stimuli (A/V s) were mixed and presented via auditory and visual modes simultaneously to test for a serial position effect. The results for both modality switching and stimulus type switching indicated a typical serial position effect, including primacy and recency effects.
INTRODUCTION
OVERVIEW
The Serial Position Effect (SPE) shows that, when given a list of items to remember, individuals freely recall a greater percentage of items from the beginning (a primacy effect) and from the end (a recency effect) compared to the percentage of items recalled from the middle of the list. This effect takes a U-shape when the percentage of recalled items is graphed as a function of item position. This serial position function is a robust finding and has had a considerable impact on the study of human memory. Research has concluded that the serial position effect occurs when the full list of items is either visual or auditory. This study continues testing of the serial position effect to see if the effect still applies when each item for a given list is presented both visually and aurally, and/or when items for a given list are either presented visually or aurally. It further continues to explore whether a serial position effect can exist for a given list of items when each item is presented as either a verbal or spatial representation.
LITERATURE REVIEW
Research findings of the serial position effect are abundant and date back to the early twentieth century. The earliest findings show that the SPE is found in freely recalling lists of nonverbal visual stimuli (Ebbinghaus, 1913). However, SPE findings seem to be dependent on how the ‘nonverbal’ visual stimulus is defined. While Ebbinghaus considered nonsensical word items as nonverbal stimuli (items which still contained letters and some were therefore still able to be sounded out), other researchers interpret the term ‘nonverbal’ as meaning purely spatial, such as Petlick’s study of SPE for multiple modalities using a series of snowflake images for nonverbal visual stimuli (1962). In the latter study, nonverbal visual stimuli was found to not show a SPE, although it was acknowledged that more research was needed to look at other kinds of‘nonverbal’ visual stimuli. In more recent studies, however, a SPE is found in recalling visual spatial stimuli, resembling the serial position curve also evident with both visual and auditory verbal items (Morris et al, 1995).
Research findings of the serial position effect are abundant and date back to the early twentieth century. The earliest findings show that the SPE is found in freely recalling lists of nonverbal visual stimuli (Ebbinghaus, 1913). However, SPE findings seem to be dependent on how the ‘nonverbal’ visual stimulus is defined. While Ebbinghaus considered nonsensical word items as nonverbal stimuli (items which still contained letters and some were therefore still able to be sounded out), other researchers interpret the term ‘nonverbal’ as meaning purely spatial, such as Petlick’s study of SPE for multiple modalities using a series of snowflake images for nonverbal visual stimuli (1962). In the latter study, nonverbal visual stimuli was found to not show a SPE, although it was acknowledged that more research was needed to look at other kinds of‘nonverbal’ visual stimuli. In more recent studies, however, a SPE is found in recalling visual spatial stimuli, resembling the serial position curve also evident with both visual and auditory verbal items (Morris et al, 1995).
Research findings by Wickens on the visual and auditory modalities suggests that the visual modality is best paired with a spatially coded display format compared to the verbal coding suggested to be best paired with the auditory modality (Wickens, 2000). Our first trial combines auditory and visual stimuli for each verbal item. Although the printed verbal stimulus presented to the visual modality is not considered to be an optimal pair, as shown in Figure 2, we expected the visual verbal stimulus to facilitate retention of the matched auditory verbal stimuli, as studies on the bimodal pair show this to be the case (Hecht & Reiner, 2008). Therefore, we expected the serial position effect to still apply in this condition.
For our second block, however, we expected that the shuffling between the two stimulus modalities would change the results of the serial position effect, as now the visual verbal stimulus (not suggested to be an optimal pair) is presented by itself without the help of an auditory verbal stimulus. As we mentioned before, the serial position effect is still found for visual verbal stimuli. However, we still expected that, when matched against other items presented verbally to the auditory modality within the same list, the concurrent task of switching between modalities as opposed to having the benefit of both simultaneously will interfere with typical SPE results.
HYPOTHESES
General: When either the modality or display codes are shuffled in a list of items, no or less significant SPEs are expected be found.
Block 1: Av + Vv paired stimuli in block 1 would result in similar findings as previous studies in regards to the serial position effect.
Block 2: Items being presented either as Av or Vv stimuli in block 2 will result in the percentage of items recalled being more a function of modality and less that of position. In other words,participants would pay attention more to either auditory or visual stimuli as opposed to both in an attempt to more easily retain words. The switching between between modalities is considered a concurrent task that would interfere with the articulatory rehearsal if stimuli presented to one modality or the other is ignored.
PARTICIPANTS
A total of 13 graduate students (8 males and 5 females) at the State University of New York at Buffalo participated the experiment in a group setting. This group of participants consisted of both native english speaking students and international students.
APPARATUS AND STIMULI
Presentations of auditory and/or visual items were given using a laptop computer connected to a projector for participants to hear and/or see stimuli. Items (object names with no longer than eight letters and three syllables) were presented to participants in the form of three stimuli: Auditory verbal, Visual verbal, and Visual spatial. The Auditory verbal (Av) stimulus presented the item as a spoken word. For example, participants would hear the word “Key”. The Visual verbal (Vv) stimulus presented the written word of the item on the overhead display. For example, participants would see the word “Key”. The Visual spatial (Vs) stimulus presented a simple graphic of the item on the overhead display whose name was simultaneously spoken. For example, participants would see an image of a key while hearing the word “key”.
note: object names with no longer than eight letters and three syllables were used for several reasons:
1. The serial position effect applies to STM, which limits one's time and ability to rehearse. The ratio of word length to number of items presented is important to control for this. All words in a set should be of a consistent length, as varying word lengths could also become a confounding variable in testing for a serial position effect.
2. Auditory vs. visual representation of the word is more accurately tested if the presentation of items in either mode take similar times to process. It is better to be consistent by testing words that are shorter to say and thus presented at the same instant and duration as the visual stimulus.
3. There is no concern with with the unfamiliarity of words being a confounding variable when words are basic/common enough to include both American and international participants.
DESIGN
In the current experiment, each subject participated in 3 blocks. In the first block, a within-subjects design was conducted to test the baseline of reaction to serial position with each verbal stimulus displayed in both auditory and visual (A/V v) channels. The primary position denotes the first four locations of items on the list; the recency position denotes last four location of items on the list, whereas the middle position referred to the remaning positions in the middle of the stimulus list.
In the second and third blocks, two 2×3 within-subjects design were adopted with stimulus modality and serial position, and stimulus type and serial position respectively. In the second block, the verbal stimulus was presented either aurally (Av) or visually (Vv); while in the third block, the stimulus presented by both visual and auditory channels was either verbal (A/V v) or spatial (A/V s). Within each of these two blocks, two trials were designed so that the orders of stimulus modality and stimulus type were counterbalanced.
Findings by Baddeley suggest that two codes do not compete for the same limited processing resources or attention (1986). In other words, time sharing occurs easily when both verbal and spatial codes are used as they do not interfere with each other. However, we hypothesized that a typical SPE would not apply if the verbal and spatial coded stimuli are not presented simultaneously but rather shuffled as are the stimulus modalities in block two.
HYPOTHESES
General: When either the modality or display codes are shuffled in a list of items, no or less significant SPEs are expected be found.
Block 1: Av + Vv paired stimuli in block 1 would result in similar findings as previous studies in regards to the serial position effect.
Block 2: Items being presented either as Av or Vv stimuli in block 2 will result in the percentage of items recalled being more a function of modality and less that of position. In other words,participants would pay attention more to either auditory or visual stimuli as opposed to both in an attempt to more easily retain words. The switching between between modalities is considered a concurrent task that would interfere with the articulatory rehearsal if stimuli presented to one modality or the other is ignored.
Block 3: Similar to the effect of switching modalities, switching display codes visually in block 3 between verbal and spatial stimuli, Vv and Vs respectively, would result in the percentage of items recalled being more a function of stimulus type than of position.
METHOD
A total of 13 graduate students (8 males and 5 females) at the State University of New York at Buffalo participated the experiment in a group setting. This group of participants consisted of both native english speaking students and international students.
APPARATUS AND STIMULI
Presentations of auditory and/or visual items were given using a laptop computer connected to a projector for participants to hear and/or see stimuli. Items (object names with no longer than eight letters and three syllables) were presented to participants in the form of three stimuli: Auditory verbal, Visual verbal, and Visual spatial. The Auditory verbal (Av) stimulus presented the item as a spoken word. For example, participants would hear the word “Key”. The Visual verbal (Vv) stimulus presented the written word of the item on the overhead display. For example, participants would see the word “Key”. The Visual spatial (Vs) stimulus presented a simple graphic of the item on the overhead display whose name was simultaneously spoken. For example, participants would see an image of a key while hearing the word “key”.
note: object names with no longer than eight letters and three syllables were used for several reasons:
1. The serial position effect applies to STM, which limits one's time and ability to rehearse. The ratio of word length to number of items presented is important to control for this. All words in a set should be of a consistent length, as varying word lengths could also become a confounding variable in testing for a serial position effect.
2. Auditory vs. visual representation of the word is more accurately tested if the presentation of items in either mode take similar times to process. It is better to be consistent by testing words that are shorter to say and thus presented at the same instant and duration as the visual stimulus.
3. There is no concern with with the unfamiliarity of words being a confounding variable when words are basic/common enough to include both American and international participants.
DESIGN
In the current experiment, each subject participated in 3 blocks. In the first block, a within-subjects design was conducted to test the baseline of reaction to serial position with each verbal stimulus displayed in both auditory and visual (A/V v) channels. The primary position denotes the first four locations of items on the list; the recency position denotes last four location of items on the list, whereas the middle position referred to the remaning positions in the middle of the stimulus list.
In the second and third blocks, two 2×3 within-subjects design were adopted with stimulus modality and serial position, and stimulus type and serial position respectively. In the second block, the verbal stimulus was presented either aurally (Av) or visually (Vv); while in the third block, the stimulus presented by both visual and auditory channels was either verbal (A/V v) or spatial (A/V s). Within each of these two blocks, two trials were designed so that the orders of stimulus modality and stimulus type were counterbalanced.
Block 1 slides: A/Vv (Auditory verbal + Visual verbal)
Block 2 Trial A slides: Av or Vv (Auditory verbal or Visual verbal)
Block 3 Trial A slides: A/Vv or A/Vs ( [Auditory verbal + Visual verbal] or [Auditory verbal + Visual spatial] )
PROCEDURE
In each trial, participants were presented items serially every 3 seconds and asked to perform a free recall task directly following the presentation in which they individually wrote down a list of as many items as they could remember.
In block 1, participants were presented a list of 16 items. Each item was presented for 3 seconds as the spoken word simultaneously with the written word on-screen (A/V v). The total time for the presentation of stimuli, therefore, was 48 seconds.
In each trial, participants were presented items serially every 3 seconds and asked to perform a free recall task directly following the presentation in which they individually wrote down a list of as many items as they could remember.
In block 1, participants were presented a list of 16 items. Each item was presented for 3 seconds as the spoken word simultaneously with the written word on-screen (A/V v). The total time for the presentation of stimuli, therefore, was 48 seconds.
In block 2, participants were presented two trials of 16 items serially (referred to as Trial 2a and Trial 2b). Each item was presented for 3 seconds as either a spoken word (Av) or the written word on-screen (Vv). Participants performed a free recall task directly following trial 2a and another free recall task directly following trial 2b, respectively.
In block 3, as in block 2, participants were presented two trials of 16 items serially (referred to as Trial 3a and Trial 3b). However, each item was presented for 3 seconds as either a spoken word simultaneously with the written word on-screen (A/Vv) or as a spoken word simultaneously with a spatial representation of the item on-screen (A/Vs). Following a free recall task directly after trial 3a and trial 3b, participants were additionally asked to record whether each item was represented verbally or spatially by writing a “V” or “S” next to the items on their free recall list. In order to make sure participants were not given the idea to develop strategies or encouraged to do so, participants were not asked to record whether items in their free recall lists were auditory or visual in trials 2a and 2b. In other words, this eliminated the possibility that participants could learn that they should develop any strategies by block 3 if they were to expect a similar recall of stimulus type for each item recalled in future trials.
RESULTS
OVERALL
Overall accuracy of recall was analyzed using the original serial position and coded serial position. By coding the serial position of 16 items, the primary position denotes the first four locations of items on the list; the recency position denotes the last four locations of items on the list, whereas the middle position refers to the remaining positions in the middle of the stimulus list. Figure 3a shows the mean percentage of correctly recalled items at each original serial position, whereas Figure 3b shows it at a coded serial position. The typical serial position effects were revealed in both figures. In other words, the correct recall rate in the middle position of the serial curve was found to be lower than that at the beginning and the end of the list.
A one-way ANOVA was conducted with the coded positions as the independent variable and rate of correctly recalled items as the dependent variable. The effect of position on the rate of correct recall was found to be significant (F(2, 77) =10.686, p<.001). Further post-hoc tests (Tukey) indicated that the rate of correct recall was significantly higher at the beginning and end of the list compared to the middle position.
STIMULUS MODALITY
In the current study, the influence of modality on the serial position effect was explored by comparing the results of the first and second blocks. The accuracy data were analyzed with a 3 (modality: visual, auditory, visual + auditory) × 3 (serial position) analysis of variance (ANOVA). The typical serial position effect on recall accuracy was significant, controlling the stimulus modalities (F (2, 55) =7.682, p<.01). A post-hoc analysis indicated that the items at the beginning and at the end of the list had significantly higher recall accuracy compared to those at the middle of the list.
As shown in Figure 4, the overall proportion of correctly recalled items presented to the auditory modality was lower than that of both modalities, whereas the proportion of correctly recalled items presented visually was higher compared to that of both modalities. However, the result of ANOVA indicated no significant effect of modality on accuracy of recall (F (2, 55) =2.509, p>.05). A post-hoc analysis indicated that the rate of correct recall with stimuli visually displayed was marginally significantly higher than that with stimuli displayed to the auditory modality. The interaction effect of stimulus modality and serial position was not significant (F (4, 55) =0.166, p>.05).
STIMULUS TYPE
In the current study, the influence of stimulus type on the serial position effect was explored by comparing the results using two stimulus types in the third block. The correctly recalled data were analyzed with a 2 (stimulus type: verbal and spatial) × 3 (serial position) analysis of variance (ANOVA). The typical serial position effect on recall accuracy was significant, controlling the stimulus modalities (F (2, 26) =8.397, p<.01). A post-hoc analysis indicated that the items at the beginning and at the end of the list had significantly higher recall accuracy compared to those at the middle of the list.
As shown in figure 5, the overall proportion of correctly recalled items presented as a verbal stimulus was only slightly lower than that of the spatial stimulus at both the beginning and end of the list. Upon the middle position of the list, however, the difference in verbal and spatial proportions is much greater, with items presented verbally still lower than spatial. The result of ANOVA indicated no significant effect of stimulus type on accuracy of recall (F (1, 26) =0.849, p>.05). The interaction effect of stimulus type and serial position was not significant (F (2, 26) =0.272, p>.05).
DISCUSSION
OVERALL FINDINGS
This experiment examined the generality of the serial position effect across two modalities (visual vs. auditory) and two stimulus types (verbal vs. spatial). Results demonstrated a general serial position effect including both primacy and recency effects in a free recall task. In other words, rates of accurately recalled items at the beginning and end of list positions were higher compared to that of items at the middle of the list. The serial position effects in both the visual and auditory modalities, as well as verbal and spatial stimulus types, are shown in Table 1.
Table 1. The percentage of correctly recalled items for each modality, stimulus type, and coded serial position category
Compared to other studies, we used a relatively large number of stimuli for each serial list and found a more significant serial position effect in doing so. By coding the original serial position, sixteen positions were categorized into three groups (primary position, middle position, and recency position). Both figure 3a (original serial position) and 3b (coded serial position) indicate the primary and recency effects of the serial position effect. According to the early two-store theory of the serial position effect suggested by Glanzer and Cunitz (1966), the primacy effect occurred because the items appearing early in the list had been stored in long-term memory, whereas the recency effect was a function of short-term memory, where items appearing at the end of the list are still stored.
In the first block of this study, we presented the verbal stimuli in the auditory and visual modes simultaneously and found the serial position effect as we hypothesized. Since previous studies had verified this effect on verbal stimuli in the auditory and visual modalities, respectively, it was easy to suggest that the serial position effect would still apply when we presented the stimuli with combined modalities.
STIMULUS MODALITY
The primacy and recency effects within the auditory modality were obtained from the results, which is consistent with findings reported by Petlick (2003). In Petlick’s experiment, nonverbal auditory stimuli with no corresponding visual stimuli were presented. In the current study, we used verbal stimulus in the auditory mode (Av), visual mode (Vv), and auditory with corresponding visual stimuli (A/Vv) to compare the stimuli modality effect. In the first block, the verbal stimuli were displayed in both auditory and visual modes while participants were presented verbal stimuli either in the auditory mode or in the visual mode in the second block. A significant typical serial position effect was found on recall accuracy (F (2, 55) =7.682, p<.01).
Moreover, as shown in figure 4, the rates of correctly recalled items presented to the auditory mode is lower than that of both modalities, whereas the rates for the visual mode is higher compared to that of both modalities. This is consistent with research indicating that there is a visual dominance effect of the bimodal visual-auditory pair (Hecht & Reiner, 2008). However, the influence of modality is only marginally significant on the overall proportion of items correctly recalled. There are findings from Wickens (2000) illustrating that the verbal code is best presented to the auditory modality as speech than to the visual modality as print regarding human working memory. The opposite result obtained from the current experiment with the stimuli presented by mixing the two modalities still needs further testing to find out if the performance is better for the visual mode than auditory mode, as the sample size in the present study is fairly small.
STIMULUS TYPE
The primacy and recency effects across stimulus type (verbal and spatial) were obtained from the results, which is consistent with findings reported by (Jones, D. et al., 1995). In their experiment, verbal and spatial stimuli were separately presented to the auditory and visual modes. In the current study, we mixed verbal stimuli (A/V v) and spatial stimuli (A/V s) and presented those stimuli by auditory and visual means simultaneously to examine whether the serial position effect existed or not. In other words, participants were presented either a verbal stimulus or spatial stimulus in both modes. A significant typical serial position effect was found on recall accuracy (F (2, 26) =8.397, p<.01), which means that the items at the beginning and at the end of the list had significantly higher recall accuracy compared to those from the middle of the list.
In addition, as shown in figure 5, the rate of correctly recalled items presented as a spatial stimulus is higher than that of verbal stimuli across the item position. However, the results of ANOVA indicated no significant effect of stimulus type on accuracy of recall (F (1,26) =0.849, p>.05). This stable difference between verbal and spatial stimuli recall may be due to the distinctive feature of the spatial stimulus over a less apparent verbal stimulus. Further studies are still necessary to explore whether there is a consistent difference between verbal and spatial stimulus recall, as there are many different levels of spatial stimuli to test against verbal stimuli and the sample size for this study is fairly small.
LIMITATIONS
While the current study validated the serial position effect across different modalities (auditory vs.visual) and different stimuli type (verbal vs. spatial), there are still several problems and limitations which make it difficult to conclude that these findings are entirely reliable. The current study was conducted using a small sample of students tested as a group, which may lead to various sources of interference. During the formal experiment, participants were asked to test individually in a controlled environment. In addition, since there was a limitation of the current experiment condition, we only used the rate of correct recall at each position of the list as the dependent variable rather than recording the reaction time and error rate, which could have provided more accurate data for recall performance. Since all participants were tested as a group, we only used the free recall task to test for a SPE in every block, which may have limited our results, as other tasks may have been better suited for one block over another. For instance, a cued recall task for the third block may have been more appropriate for evaluating visual-spatial stimuli.
REFERENCES
Baddeley,A. D. (1986). Working memory. Oxford: Clarendon.
Ebbinghaus, Hermann (1913). On memory: A contribution to experimental psychology. New York: TeachersCollege.
Hecht, D., Reiner, M. (2008) Sensory dominance in combination of audio, visual, and haptic stimuli. Exp Brain Res, 193,3047-314.
Jones, D., Farrand, P., Stuart, G., &Morris, N. (1995). Functional equivalence of verbal and spatial information in serial short-term memory. Journal Of Experimental Psychology: Learning,Memory, And Cognition, 21(4), 1008-1018
Petlick, J. H. (2003). Learning and memory in the visual, auditory, and olfactory modalities: An investigation of the generality of serial position effects. North Carolina State University. ProQuest Dissertations and Theses, 71 p.
Wickens, C. D., & Hollands, J. G. (2000). Engineering psychology and human performance (3rd ed.). Upper Saddle River, NJ:Prentice Hall.
ORIANE DALEY, Y. ZHANG IE 532 Human Information Processing Spring 2012
